{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Iterators and Iterables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Previously we saw that we could create **iterator** objects by simply implementing:\n",
    "\n",
    "* a `__next__` method that returns the next element in the container\n",
    "* an `__iter__` method that just returns the object itself (the iterator object)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Doing that we could use a `for` loop, list comprehensions, and in fact use that iterator object anywhere an iterable was expected (like `enumerate`, `sorted`, and so on)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, we had two outstanding issues/questions:\n",
    "* when we looped over the iterator using a `for` loop (or a comprehension, or other functions that do some form of iteration), we saw that the `__iter__` was always called first.\n",
    "* the iterator gets exhausted after we have finished iterating it fully - which means we have to create a new iterator every time we want to use a new iteration over the collection - can we somehow avoid having to remember to do that every time?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The answer to both of these questions are related."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start by looking at how we might avoid having to create a new instance of the collection every time we want to iterate over it.\n",
    "\n",
    "After all, we don't need a new instance of the elements, just some kind of *resetting* of *current* item."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's start with a simple example that has those issues:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Cities:\n",
    "    def __init__(self):\n",
    "        self._cities = ['Paris', 'Berlin', 'Rome', 'Madrid', 'London']\n",
    "        self._index = 0\n",
    "    \n",
    "    def __iter__(self):\n",
    "        return self\n",
    "    \n",
    "    def __next__(self):\n",
    "        if self._index >= len(self._cities):\n",
    "            raise StopIteration\n",
    "        else:\n",
    "            item = self._cities[self._index]\n",
    "            self._index += 1\n",
    "            return item"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we have an **iterator** object, but we need to re-create it every time we want to start the iterations from the beginning:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(0, 'Paris'), (1, 'Berlin'), (2, 'Rome'), (3, 'Madrid'), (4, 'London')]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities = Cities()\n",
    "list(enumerate(cities))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['PARIS', 'BERLIN', 'ROME', 'MADRID', 'LONDON']"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities = Cities()\n",
    "[item.upper() for item in cities]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Berlin', 'London', 'Madrid', 'Paris', 'Rome']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities = Cities()\n",
    "sorted(cities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, we basically have to \"restart\" an iterator by **creating a new one each time**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But in this case, we are also re-creating the underlying data every time - seems wasteful!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Instead, maybe we can split the **iterator** part of our code from the **data** part of our code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Cities:\n",
    "    def __init__(self):\n",
    "        self._cities = ['New York', 'Newark', 'New Delhi', 'Newcastle']\n",
    "        \n",
    "    def __len__(self):\n",
    "        return len(self._cities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's create our iterator this way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class CityIterator:\n",
    "    def __init__(self, city_obj):\n",
    "        # cities is an instance of Cities\n",
    "        self._city_obj = city_obj\n",
    "        self._index = 0\n",
    "        \n",
    "    def __iter__(self):\n",
    "        return self\n",
    "    \n",
    "    def __next__(self):\n",
    "        if self._index >= len(self._city_obj):\n",
    "            raise StopIteration\n",
    "        else:\n",
    "            item = self._city_obj._cities[self._index]\n",
    "            self._index += 1\n",
    "            return item"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So now we can create our `Cities` instance **once**:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cities = Cities()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "and create as many iterators as we want, but passing it the same `Cities` instance everyt time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iter_1 = CityIterator(cities)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "New York\n",
      "Newark\n",
      "New Delhi\n",
      "Newcastle\n"
     ]
    }
   ],
   "source": [
    "for city in iter_1:\n",
    "    print(city)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['NEW YORK', 'NEWARK', 'NEW DELHI', 'NEWCASTLE']"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "iter_2 = CityIterator(cities)\n",
    "[city.upper() for city in iter_2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, we're almost at a solution now. At least we can create the **iterator** objects without having to recreate the `Cities` object every time.\n",
    "\n",
    "But, we still have to remember to create a new iterator, **and** we can no longer iterate over the `cities` object anymore!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'Cities' object is not iterable",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-11-d57230a6a2ef>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[1;32mfor\u001b[0m \u001b[0mcity\u001b[0m \u001b[1;32min\u001b[0m \u001b[0mcities\u001b[0m\u001b[1;33m:\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m\u001b[0;32m      2\u001b[0m     \u001b[0mprint\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0mcity\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n",
      "\u001b[1;31mTypeError\u001b[0m: 'Cities' object is not iterable"
     ]
    }
   ],
   "source": [
    "for city in cities:\n",
    "    print(city)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is where the first question we asked comes into play. Whenever we iterated our iterator, the first thing Python did was call `__iter__`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In fact, let's just check that again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class CityIterator:\n",
    "    def __init__(self, city_obj):\n",
    "        # cities is an instance of Cities\n",
    "        print('Calling CityIterator __init__')\n",
    "        self._city_obj = city_obj\n",
    "        self._index = 0\n",
    "        \n",
    "    def __iter__(self):\n",
    "        print('Calling CitiyIterator instance __iter__')\n",
    "        return self\n",
    "    \n",
    "    def __next__(self):\n",
    "        print('Calling __next__')\n",
    "        if self._index >= len(self._city_obj):\n",
    "            raise StopIteration\n",
    "        else:\n",
    "            item = self._city_obj._cities[self._index]\n",
    "            self._index += 1\n",
    "            return item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling CityIterator __init__\n"
     ]
    }
   ],
   "source": [
    "iter_1 = CityIterator(cities)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling CitiyIterator instance __iter__\n",
      "Calling __next__\n",
      "New York\n",
      "Calling __next__\n",
      "Newark\n",
      "Calling __next__\n",
      "New Delhi\n",
      "Calling __next__\n",
      "Newcastle\n",
      "Calling __next__\n"
     ]
    }
   ],
   "source": [
    "for city in iter_1:\n",
    "    print(city)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Iterables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we finally come to how an **iterable** is defined in Python.\n",
    "\n",
    "An **iterable** is an object that:\n",
    "* implements the `__iter__` method\n",
    "* and that method returns an **iterator** which can be used to iterate over the object"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What would happen if we put an `__iter__` method in the `Cities` object and then try to iterate?\n",
    "\n",
    "When we try to iterate over the `Cities` instance, Python will first call `__iter__`. The `__iter__` method should then return an **iterator** which Python will use for the iteration.\n",
    "\n",
    "We actually have everything we need to now make `Cities` an **iterable** since we already have the `CityIterator` created:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class CityIterator:\n",
    "    def __init__(self, city_obj):\n",
    "        # cities is an instance of Cities\n",
    "        print('Calling CityIterator __init__')\n",
    "        self._city_obj = city_obj\n",
    "        self._index = 0\n",
    "        \n",
    "    def __iter__(self):\n",
    "        print('Calling CitiyIterator instance __iter__')\n",
    "        return self\n",
    "    \n",
    "    def __next__(self):\n",
    "        print('Calling __next__')\n",
    "        if self._index >= len(self._city_obj):\n",
    "            raise StopIteration\n",
    "        else:\n",
    "            item = self._city_obj._cities[self._index]\n",
    "            self._index += 1\n",
    "            return item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Cities:\n",
    "    def __init__(self):\n",
    "        self._cities = ['New York', 'Newark', 'New Delhi', 'Newcastle']\n",
    "        \n",
    "    def __len__(self):\n",
    "        return len(self._cities)\n",
    "    \n",
    "    def __iter__(self):\n",
    "        print('Calling Cities instance __iter__')\n",
    "        return CityIterator(self)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cities = Cities()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n",
      "New York\n",
      "Calling __next__\n",
      "Newark\n",
      "Calling __next__\n",
      "New Delhi\n",
      "Calling __next__\n",
      "Newcastle\n",
      "Calling __next__\n"
     ]
    }
   ],
   "source": [
    "for city in cities:\n",
    "    print(city)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And watch what happens if we try to run that loop again:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n",
      "New York\n",
      "Calling __next__\n",
      "Newark\n",
      "Calling __next__\n",
      "New Delhi\n",
      "Calling __next__\n",
      "Newcastle\n",
      "Calling __next__\n"
     ]
    }
   ],
   "source": [
    "for city in cities:\n",
    "    print(city)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A new **iterator** was created when the `for` loop started.\n",
    "\n",
    "In fact, same happens for anything that is going to iterate our iterable - it first calls the `__iter__` method of the itrable to get a **new** iterator, then uses the iterator to call `__next__`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[(0, 'New York'), (1, 'Newark'), (2, 'New Delhi'), (3, 'Newcastle')]"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(enumerate(cities))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "['Newcastle', 'Newark', 'New York', 'New Delhi']"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sorted(cities, reverse=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can put the iterator class inside our `Cities` class to keep the code self-contained:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "del CityIterator  # just to make sure CityIterator is not in our global scope"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Cities:\n",
    "    def __init__(self):\n",
    "        self._cities = ['New York', 'Newark', 'New Delhi', 'Newcastle']\n",
    "        \n",
    "    def __len__(self):\n",
    "        return len(self._cities)\n",
    "    \n",
    "    def __iter__(self):\n",
    "        print('Calling Cities instance __iter__')\n",
    "        return self.CityIterator(self)\n",
    "    \n",
    "    class CityIterator:\n",
    "        def __init__(self, city_obj):\n",
    "            # cities is an instance of Cities\n",
    "            print('Calling CityIterator __init__')\n",
    "            self._city_obj = city_obj\n",
    "            self._index = 0\n",
    "\n",
    "        def __iter__(self):\n",
    "            print('Calling CitiyIterator instance __iter__')\n",
    "            return self\n",
    "\n",
    "        def __next__(self):\n",
    "            print('Calling __next__')\n",
    "            if self._index >= len(self._city_obj):\n",
    "                raise StopIteration\n",
    "            else:\n",
    "                item = self._city_obj._cities[self._index]\n",
    "                self._index += 1\n",
    "                return item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cities = Cities()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n",
      "Calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[(0, 'New York'), (1, 'Newark'), (2, 'New Delhi'), (3, 'Newcastle')]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "list(enumerate(cities))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Technically we can even get an iterator instance ourselves directly, by calling `iter()` on the `cities` object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n"
     ]
    }
   ],
   "source": [
    "iter_1 = iter(cities)\n",
    "iter_2 = iter(cities)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, Python created and returned two different instances of the `CityIterator` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1741231353928, 1741231354320)"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id(iter_1), id(iter_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And now we also have should understand why **iterators** also implement the `__iter__` method (that just returns themselves) - it makes them **iterables** too!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Mixing Iterables and Sequences"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Cities` is an iterable, but it is not a sequence type:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cities = Cities()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(cities)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "ename": "TypeError",
     "evalue": "'Cities' object does not support indexing",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-30-e8dda3d9f35c>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mcities\u001b[0m\u001b[1;33m[\u001b[0m\u001b[1;36m1\u001b[0m\u001b[1;33m]\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mTypeError\u001b[0m: 'Cities' object does not support indexing"
     ]
    }
   ],
   "source": [
    "cities[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our Cities **could** also be a sequence, we could also decide to implement the `__getitem__` method to make it into a sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "class Cities:\n",
    "    def __init__(self):\n",
    "        self._cities = ['New York', 'Newark', 'New Delhi', 'Newcastle']\n",
    "        \n",
    "    def __len__(self):\n",
    "        return len(self._cities)\n",
    "    \n",
    "    def __getitem__(self, s):\n",
    "        print('getting item...')\n",
    "        return self._cities[s]\n",
    "    \n",
    "    def __iter__(self):\n",
    "        print('Calling Cities instance __iter__')\n",
    "        return self.CityIterator(self)\n",
    "    \n",
    "    class CityIterator:\n",
    "        def __init__(self, city_obj):\n",
    "            # cities is an instance of Cities\n",
    "            print('Calling CityIterator __init__')\n",
    "            self._city_obj = city_obj\n",
    "            self._index = 0\n",
    "\n",
    "        def __iter__(self):\n",
    "            print('Calling CitiyIterator instance __iter__')\n",
    "            return self\n",
    "\n",
    "        def __next__(self):\n",
    "            print('Calling __next__')\n",
    "            if self._index >= len(self._city_obj):\n",
    "                raise StopIteration\n",
    "            else:\n",
    "                item = self._city_obj._cities[self._index]\n",
    "                self._index += 1\n",
    "                return item"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "cities = Cities()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's a sequence:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "getting item...\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'New York'"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cities[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's also an iterable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'New York'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter(cities))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now that Cities is both a sequence type (`__getitem__`) and an iterable (`__iter__`), when we loop over `cities`, is Python going to use `__getitem__` or `__iter__`?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Calling Cities instance __iter__\n",
      "Calling CityIterator __init__\n",
      "Calling __next__\n",
      "New York\n",
      "Calling __next__\n",
      "Newark\n",
      "Calling __next__\n",
      "New Delhi\n",
      "Calling __next__\n",
      "Newcastle\n",
      "Calling __next__\n"
     ]
    }
   ],
   "source": [
    "cities = Cities()\n",
    "for city in cities:\n",
    "    print(city)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It uses the iterator - so Python will use the iterator if there is one, otherwise it will fall back to using `__getitem__`. If neither is implemented, we'll get an exception.\n",
    "\n",
    "Of course, for selection by index or slice, the `__getitem__` method **must** be implemented.\n",
    "\n",
    "We'll come back to this very topic in an upcoming video, because behind the scenes, even if we only implement the `__getitem__` method, Python will auto-generate an iterator for us!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Built-In Iterables and Iterators"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The way iterables and iterators work in our custom `Cities` example is exactly the way Python iterables work too."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "l = [1, 2, 3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since lists are iterables, they implement the `__iter__` method and we can get an **iterator** for the list:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iter_l = iter(l)\n",
    "#or could use iter_1 = l.__iter__()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "list_iterator"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(iter_l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter_l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter_l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter_l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "ename": "StopIteration",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m                             Traceback (most recent call last)",
      "\u001b[1;32m<ipython-input-42-c2f95d80ff72>\u001b[0m in \u001b[0;36m<module>\u001b[1;34m()\u001b[0m\n\u001b[1;32m----> 1\u001b[1;33m \u001b[0mnext\u001b[0m\u001b[1;33m(\u001b[0m\u001b[0miter_l\u001b[0m\u001b[1;33m)\u001b[0m\u001b[1;33m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[1;31mStopIteration\u001b[0m: "
     ]
    }
   ],
   "source": [
    "next(iter_l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "See? The same `StopIteration` exception is raised.\n",
    "\n",
    "Since `iter_l` is an iterator, it also implements the `__iter__` method, which just returns the iterator itself:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(1741231347248, 1741231347248)"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "id(iter_l), id(iter(iter_l))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__next__' in dir(iter_l)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__iter__' in dir(iter_l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since the list `l` is an iterable it also implements the `__iter__` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__iter__' in dir(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "but does not implement a `__next__` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__next__' in dir(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, since lists are also sequence types, they also implement the `__getitem__` method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__getitem__' in dir(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Sets and dictionaries on the other hand are not sequence types:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__getitem__' in dir(set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__iter__' in dir(set)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = {1, 2, 3}\n",
    "'__next__' in dir(iter(s))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'__iter__' in dir(dict)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But what does the iterator for a dictionary actually return? It iterates over what? You should probably already guess the answer to that one!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "d = dict(a=1, b=2, c=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iter_d = iter(d)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'a'"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter_d)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Dictionary iterators will iterate over the **keys** of the dictionary."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To iterate over the values, we could use the `values()` method which returns an **iterable** over the values of the dictionary:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iter_vals = iter(d.values())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter_vals)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And to iterate over both the keys and values, dictionaries provide an `items()` iterable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "iter_items = iter(d.items())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "('a', 1)"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "next(iter_items)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we get an iterator over key, value tuples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll examine the usefullness of being able to iterate using `next` instead of a `for` loop, or comprehension, in the next video."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
